Characterizing one-class datasets

This paper aims at characterizing classification problems to find the main features that determine the differences in performance by different classifiers. It is known that, using the disagreements between the classifiers, a distance measure between datasets can be defined. The datasets can then be embedded and visualized in a 2-D scatterplot. This embedding thus reveals the structure of the set of problems. In this paper we focus on a specific pattern recognition problem, the problem of outlier detection or one-class classification, where classifiers have to detect if a new object resembles the training data or not. For this problem the outputs of many classifiers on many datasets are available. By inspecting the scatterplot of the datasets, two main features appear to characteristize the datasets; (1) their effective sample size and (2) the class overlap. By generating artificial datasets for which these variables are varied, these observations are confirmed experimentally.